Model-Based Reinforcement Learning with Multinomial Logistic Function Approximation

We study model-based reinforcement learning (RL) for episodic Markov decision processes (MDP) whose transition probability is parametrized by an unknown core with features of state and action. Despite much recent progress in analyzing algorithms the linear MDP setting, understanding more general models very restrictive. In this paper, we propose a provably efficient RL algorithm given multinomial logistic model. show that our proposed based on upper confidence bounds achieves O(d√(H^3 T)) regret bound where d dimension core, H horizon, T total number steps. To best knowledge, first function approximation provable guarantees. also comprehensively evaluate numerically it consistently outperforms existing methods, hence achieving both efficiency practical superior performance.